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CHAPTER I 



INTRODUCTIOiN' 

Both the quiz and knowledge of results as instructional aids have 

I 

been topics of studies dating back four decades. The two areas of in- 
vestigation have proceeded separately, however, with few studies eing 
concerned with the effects of both of these variables or their interaction. 

i 

This state of affairs is surprising for two reasons: (1) knowledge of 

results or feedback has consistently been utilized with reference to the 
quiz; (2) both variables have been hypothesized to influence the motiva- 
tion of the subject and to provide structure and practice on the material. 

Recently there has been a focus' on the behavior in which the 
subject engages while learning. Both the quiz and delayed knowledge of 
results are thought to enhance the facilitative effects of these behaviors 
on learning verbally mediated materials. Here again the investigations 
have proceeded separately and one cannot ascertain whether or not delayed 
knowledge of results would elicit or reinforce facilitative learning 
behavior if the quiz has already done so. The question then is to deter- 
mine whether the effects of the quiz and knowledge of results are 
additive or redundant. 

To complicate the picture further, there may be individuals whos'e 
achievement, because of their reliance on more extrinsic immediate evalu.- 
ative cues, would be enhanced more by immediate feedback. Others who 
utilize their own intrinsic cues may indulge in the facilitative behavior 
which the quiz and delayed knowledge of results are thought to influence. 



. 2 



ThuSj learning may be enhanced differentially in persons of different 
achievement orientations. 

Whatever the effects of the quiz or knowledge of results on learn- 
ing, it appears to be more effective if it is introduced during the ac- 
quisition process, that is, close to the time the materiad"^ is first pre- 
sented (Spitzer 1939; Sonces & Stroud 1940; Angell 1949; Pressey 1950; 
Skinner 1954, 1958; Baker, Schultz, Sullivan & Morgan 1966). It is at 

I 

this time that the learner supposedly enga^ges in behaviors which facili- 
tate learning. According to Brackbill (Brackbill, Bravos & Starr 1962; 
Brackbill, Isaacs & Smelkinson 1962; Brackbill & Kappy 1962; Brackbill, 
Wagner & Wilson 1964), the learner utilizes mediational responses during 
the delay which in turn are reinforced by the delayed knowledge of re- 
sults. Since Brackbill 's studies were verbal discrimination tasks, the 
quiz was, of course, inherent in the presentation of the material. 

English and Kinzer (1966) used reading material arid gave a quiz immedi- 
ately after the reading was completed. Delay of knowledge of results on 
the quiz was then varied. What is important here is that the quiz was 
not inherent in the presentation of the material. 

Rothkopf's (unpublished paper, Rothkopf 1965; paradigm for the 
effects of the quiz was similar to that of the delay of feedback studies. 
Here again the quiz was thought to reinforce the facilitative behaviors 
engaged in by the subject while learning. But, in contrast to the English 
and Kinzer study the 
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quiz was built into the reading of the material. Also, delay of 
the quiz from the reading proper was not varied but subjects were 
able to control this themselves. In addition, Rothkopf surmised 
that one could manipulate the material in such a way as to modify or 
alter these facilitative behaviors. The three paradigms are illus- 

. / ' V ^ 

trated below: ' • 



effective 

stimuli 



segmentation 
attention 

translating into internal. speech 
mediating responses 
mnemotechnic devices 
rehearsal or review 
Rothkopf: read material/ quiz / read more material 

preventio n pf coupling of unrelate 1 events 
coupling of related events 
mediational responses 
^practice 

Brackbill: presentation/ S's response / delay/ KR / next presentation 
English and Kinzer: read material/ quiz / delay / KR 

The description of the behaviors engaged in right before or during 
the critical variable in all three paradigms are similar and include 
practice on the material, mnemotechnic devices, seeing relationships 
in the material, mediational responses, attention, and segmentation. 

One major difference, besides the fact that Rothkopf did not delay the 
quiz as the others did knowledge of results, is the criterion used. 

Whereas the delay studies used the same questions as the original quiz,. 
Rothkopf *s interest was in the generaMzation of the facilitative 

behaviors, to non -repeat questions over the same material. In other 

* ^ 

words, do these mediational behaviors generalize to other material 
learned but not tested during acquisition? Note that in the Brackbill 
and Rothkopf paradigms, the behaviors evaluated or reinforced can be 
utilized during further presentation but not in the English and Kinzer 



ERIC 



4 



V 



paradigm. 

Since the studies concerned with mathemagenic behaviors and 
delay of knowledge of results have been with reading materials the 
question arises whether these results can be generalized to the lecture 
method used in the classroom setting. Further, the question, implied 
above, as to the similarity or sameness of the behaviors in the delay 
of knowledge studies and the Rothkopf studies has not been scrutinized. 

If they are tapping the same processes, then the opportunity which the 
subject has during learning to engage in these behaviors would be crucial. 
Thus, short delays between the presentation of the material and the quiz 
or knowledge of results would provide the opportunity for the subject 
to engage in behaviors which facilitate his learning and recall of the 
material. 

When the delay,’ be^it of the quiz or knowledge of results, is 
introduced, another important dimension to consider is whether the sub- 
ject utilizes it to engage in facilitative learning behaviors. Not only 
must the opportunity arise, but the individual must make use of the 
opportunity. Since the dependently orientated students would be expect- 
ed to rely on extrinsic guides and direction to achieve and evaluate 
their achievement (Cronbach 1967), it is possible that they would be less 
likely to engage in these facilitative behaviors. On the other hand, the 

more independently orientated student would be more likely to process 

§ 

material during the delay, or in other words, would not rely on the ex- 
trinsic direction as much. 

If the criterion for learning in a classroom setting is a final 
% 

examination which repeats no questions* from quizzes given daily through- 
out the course, the following hypotheses are offered: 
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HYPOTHESIS 1: The effects tapped in dela)j^*of knowledge of 

results and use of the quiz *^ith reading materials are ge^ralizable 
to the lecture method. That is, if the person has the opportunity to 
engage in mathemagenic behaviors while learning from the lecture method 
of instruction, his final examination score vzill be enhanced. 

HYPOTHESIS 2: Behaviors engaged in during the delay of knowledge 

of. results are the same as those engaged in before taking a quiz. 
Participation in such behaviors will result in higher examination scores 
in either case, providing the delay of feedback is not redundant to the 
delay of the quiz. Hence, mathemagenic behaviors, whether maintained 

% 

^ * 

by delay of feedback or the quiz v;ill be generalizable to material not 
specifically tested during acquisition. 

HYPOTHESIS 3: Persons receiving a delayed quiz of one day after 

each lecture will perform better on a final examination than will those 
receiving either an immediate quiz or no quiz. This is because the 
persons receiving the delayed quiz would have opportunity to engage in 
mathemagenic behaviors which in turn would be evaluated or reinforced 
by the quiz. 

HYPOTHESIS 4: Persons receiving delayed knowledge of results on 

an immediate quiz (i.e. directly following the lecture) will, perform ^ 
better on a final examination than those receiving immediate feedback. 
Here again the subject will have had opportunity to engage in mathe- 
magenic behavior. The effects of delayed knowledge of results on a 
delayed quiz will be non-significant due to the redundancy with the 
delayed quiz. 

HYPOTHESIS 5: As persons increase in achievement via conformity 

so will performance on the final exam if they are given immediate feed- 
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back on the quiz during learning. The immedialie extrinsic direction 
inherent here is assumed to be more conducive to learning for these 

I 

SUDjucLs ♦ 

HYPOTHESIS 6: As persons increase in achievement via independ- 

ence they will perform better on a final examination under less struc- 
tured conditions including a delayed quiz, delayed feedback, and no 
feedback. The more independent achievement orientation a person has the 
more he will engage in mathemagenic behaviors and intrinsic evaluations 
conducive to learning. 
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CHAPTER TI 
METHODOLOGY 



Sub j 6ct s ; SubjGcts wgtg 294 maririGS and sailors GnrollGd in ClassGS 
708 and 715 of AMFU (A) school at tliG Momphis Air Station. This school 
was a four WGGk integrated course, meaning that different subject matter 
was being taught at different times throughout* the course. Only the 
physics classes vzere utilized in the study. Generally, one physics class 
invplving, on the average, two class hours per day was taught the last 



three of the four weeks. 

Method: Since the interest was in the effects of different types of 

treatments with the quiz and knowledge of results and their interaction 

% 

a 2 X 3 random groups factorial design was set up using final examination 
as the dependent variable. Thus, six of nine sections of each clas.s, were 
used for the treatment conditions. Another section received no quiz or 
feedback and served as a control. The quiz was varied two ways, immedi- 
ately after a lecture and delayed one day. For each variation of the 
quiz feedback was given in the form of the correct answer either immediately 
after th.e quiz or delayed one day. Another group received only the 
number right one day after taking the quiz. 



In line with hypotheses one and three it was predicted that the 






^i^^l s^smination mean of the delayed quiz sections would be significant- 
ly greater than those sections receiving an immediate quiz. Hypotheses 
two and four predicts that there would be no difference in sections 
receiving different delays ol feedback except the section receiving 
delayed feedback on an immediate quiz would be expected to have a higher 
mean. Hence, a knowledge of results by quiz interaction was predicted. 
Only the delayed quiz and immediate qui.z/dc.layed feedback sections were 



o 




die final examination than the control 



predicted to score higher on 
section if mathemagenic behavior is the crucial variable. 

To assess the differential effects of the two achievement orien- 
tation variables, a correlational technique was used (Cronbach, 1967 ). 
Since covariance of achievement orientation with mathemagenic behaviors 
was predicted, the two achievement scales were each correlated with 
criterion under different conditions. Differences in correlations were 
tested for significance using Fisher's z transformation. Because the 
achievement scales used were not orthogonal measures, a part correlation 
was used for statistical control of the effects of one on the predic- 
tion by the other. This, in effect, reduced the correlation between 

them to zero, making them orthogonal. 

Following hypothesis five. Achievement via Confomity was predicted 

to correlate higher in sections receiving immediate feedback than in 
those receiving delayed or no feedback. No difference in correlation 
was predicted between the different quiz sections or between the delayed 
feedback and no feedback sections. For hypothesis six. Achievement via 
Independence was predicted to correlate higher in sections receiving 
the delayed quiz than in those receiving an immediate quiz. Likewise, 
Achievement via Independence was expected to correlate higher in delayed 
or no feedback sections than in immediate feedback sections. No diffe- 

rence.in correlation was predicted’ for the no feedback and delayed 

» ' „ 

feedback sections. 

Measures: Five separate measures were used in the study, two of which 

were control measures in case different sections of the class were sig- 
nificantly different in general mental ability and/or initial level in 



physics. Since both sailors and marines V7cre used in the study, it 

V;as necessary to rely on different Basic Te,st Batteries (BTB) for each 

of these for a measure of mental ability. However, the Navy General 

Classif icatj.on Test (GCT) and Marine Verbal Test (VE) are similar 

measures of verbal ability and correlate .83. Likewise, the Arithmetic 

Test (ARI) of the Navy and Arithmetic Reasoning Test (ARI) of the 

marines are similar measures of numerical ability and; correlate .76. 

GCT + ARI has a distribution with a mean of 100 and a standard deviation 

of 18. ^ has about the same mean and standard deviation (B. Rimland 

2 

personal communication) . 

As a measure of initial ability a 16 item pre-test was given 
prior to the students' entry into the course. The school has three 
alternate final examinations in physics which are considered equivalent. 

The pretest was one such exam plus one additional item. The ^6 item 
criterion final examination consisted of the form actually given for the 
final in the experimental Class plus the items given on the pretest plus 
fifteen items drav;n from the third alternate fom\ and the exhaustive 
test item of the school. This criterion exam had a split half '*re- 

liability of .81. Quiz items given throughout the course were not 
repeated on the final although these items were also gleaned from the 
item pool plus a fev; constructed by the experimenter from the detailed 
lesson guide for the course. 

The measures of achievement orientation have been briefly dis- 
cussed in the review of the literature. They were the Achievement via 
Conformity (Ac) and Achievement via Independence (Ai) scales abstracted 
from Gough's California Psychological Inventory (CPI). Ac is thought 
"to identify those factors of interest and motivation which facilitate 
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achicvcviicnL Ln any sotting where confori.iance is a positive factor" 

(Gough, L957). It is a thirty-eight item scale which differentiates 
between high and low achievers in high school (Gough, 1953b), The 
items deni in optimism, diligence, acceptance of conventions, orderli- 
ness, personal efficiency, and academic effectiveness (Gough, 1949, 

1953b). Generally the scale correlates about .41 with high school grades 
(Gough, 1949, 1953b, 1957, 1964a; Gough & Fink, 1964), but fails to pre- 
dict college grades (Gough, 1949, 1953b). This predictive power appears 
to be, in part, additive to mental abilit^^as the multiple R of Ac and 

IQ with grade point average is higher than either taken along (Gough, 1949, 
1953b, 1964a). 



The Ai scale is a thirty-two item scale designed "to identify 
those factors of interest and motivation which facilitate achievement 
in any setting where autonomy and independence are positive behaviors." 
(Gough, 1957). Half of the items seem to be concerned with personal in- 
dustriousness or absence of rigidity and the other haTf seem to deal with 
the absence of fears of, and dependence on, outside forces. The scale 
generally correlates about .38 with college grades (Gough, 1953a, 1957, 
1964b; Barnette, 1961; Bendig & Klugh, 1956; Klugh & Bendig, 1955). 

Some studies indicate lower correlations with high school grades (Bendig 
& Klugh, 1956; Gough, 1964a; Gough & Fink, 1964) whereas another in- 
dicates about the same as for college grades (Gough, 1953a). Like the Ac 
scale, Ai seems to add unique variance to a multiple regression equa- 

I 

tion (Gough, 1953a, 1964b; Klugh & Bendig, 1955; Bendig & Klugh, 1956; 
Barnette, 1961; Rosenberg, McHenry, Rosenberg, & Nichols, 1962). 

^o_cedurc; Before entering the AMFU-A school, at the end of their 
preceding course, the subjects were administered the physics pretest 
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and the Ai and Ac scales. Students entering AMFU-A school from other 
sources such as the fleet, were not pretested and did not enter into 
the data collection. The latter did, however, take the quizzes and. 
final along with the rest of the class. This involved about four 
students per section. 

Subjects were assigned to their respective sections alphabetically 
by the school personnel. Each section was designated by a letter. The 
first seven sections, A through G, were used for tlie study with one 
exception. Due to the low number of students in section F of Class 708 
who had been pretested, section H was substituted for section F. The 
• treatments were as follows for Class 708: 



Section A: 


Quiz given immediately after each lesson. Number 
of correct answers given to students the next 
day. (IN) 


Section B: 


Quiz given immediately after each lesson. Correct 
answers given immediately after each quiz, (n) 


Section C: 


Quiz given immediately after each lesson. Correct 
answers given next day after quiz. (ID) 


Section D: 


Quiz given next day after lesson. Number of correct 
answers given to students the next day after quiz. (DN) 


* Section E: 


Quiz given next day after lesson. Correct answers 
given Immediately after quiz. (DI) 


Section H: 


Quiz given next day after lesson. Correct answers 
given next day after quiz. (DD) 


Section G: 


Conventional class. No quiz or feedback. (Co) 



In-Class 715 sections A and E, B and H, and D and C were switched as 

to treatment and section F substituted for section G. Subjects numbered 

133 for Class 708 and 161 for 715 giving a combined sample of 294. 

Quizzes were begun with the first physics lesson in the middle 

of tht? second week of the course. Each quiz had four to six multiple 

choice items covering the material of the lesson in question. Subjects 
*■» * 



were given five minutes to take each quiz either at the end of the 
% 

lesson for the immediate quiz or before the next day's lesson for 
the delayed quiz. Immediate feedback was given by the subjects' 
opening the left margin of the test paper which had been folded back 
and on which the correct answer was written. They then could study 
the questions and answers for five minutes. For the delayed feedback 
conditions the left margin had been severed and before the quiz was 
returned preceding the next day's lesson it was stapled to the quiz 
paper. Again five minutes were allowed to study the questions and 
answers. Subjects who received only the number correct were handed slips 
of paper the day after taking the quiz, bearing their name, quiz number, 
and the number of items correct. 

The fifteen items of the course final examination were administer- 
ed as a part of the total course final examination. The additional 
thirty-one items including those of the pretest were administered the 
next hour on the last day of the course. 



CHAPTER III 



* RESULTS 

The data were such that they could be approached two ways. 

Either the data from each class could have been analyzed separately 
as two samples or the combined sample could have been analyzed as one. 
Both approaches were taken. Data from Class 715 were thus used as a 
cross-validation of the results from Class 708. Then the samples were 
combined and an analysis was made of the total sample. 

The results of the 2x3 factorial analysis of variance for 
Class 708 are shown in Table 1. Only the difference in means of the 
quiz conditions was significant. The quiz by feedback interaction 
hypothesized did not approach significance nor was there significant 
differences among the means of the feedback groups. To assess if these 
results could be accounted for by heterogeneity of variance among the 
groups a Bartlett’s test for homogeneity was applied. That test failed 

to reach significance so it was accepted that the groups were homogeneous 

V 

in variance. 

That each section receiving the delayed quiz did better on the 
final than did those receiving an immediate quiz is seen in the results 
of Duncan's New Multiple Range test summarized in Table 2. There were 
no significant differences at the .05 level among the three delayed 
quiz groups or among the three immediate quiz groups. Such results 
follow predictions with the exception that the immediate quiz/delayed 
feedback section did not approach the level of the delayed quiz as 
predicted. Also, the delayed quiz/no feedback mean only approached 
significance over the immediate quiz groups. To assess if differences 



TABLE 1 



Results of Analysis of Variance on Final in Class 708 



Source 


SS 


df 


MS 


F 


Quiz 


457.41 


1 


457.41 


16.85* 


Feedback 


15.99 


2 


8.00 


— 


Q X F 


0.85 


2 


0.43 


— 


Within 


2931.41 


108 


27.14 





*■ Significant at .01 level. 

J 




r\ 







15 



# 










TABLE 2 

Results of Duncan's New Multiple Range Test for Class 708 

at .05 Level 



Section 



B 



H 



Condition 



IN 



ID 



II 



DN 



DD 



DI 



Mean 



31.48 



31.61 



32.00 35.00 36.05 



36.06 



Means connected by the same line are not significantly different. 
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in the treatment means with the control group mean were greater than 
that expected by chance a t test was used comparing each mean with the 
control. None of the immediate quiz means was significantly different 
than the control (IN, t=.58, df 38, p >.40; II, t=.26, df 36, p >.60; 

ID, t=^6, df 35, p >.60). Such v;as according to prediction except in 
the case of the immediate quiz/delayed feedback section. Two of the 
delayed quiz groups reached significance and the other approached it, 

(DN, t=1.31, df 36, .05 < ‘p <.10; DI , t=1.94, df 35, p <.05; DD, t=2.02, 
df 36, p <.05). 

The part correlations among the different quiz and feedback condi- 
tions are shown in Table 3 and the results of their comparisons are 
given in Table 4. Out of five comparisons in which direction was pre- 
dicted, none reversed direction. That is, r„,,.', . was higher under 

F(Ai.Ac) ^ 

the delayed quiz than the immediate, quiz and higher under no and delayed 
feedback than under immediate feedback. Likewise, Ai) higher 

in the immediate feedback sections than in the no or delayed feedback 
sections. I^/here difference was not predicted, significance was far 



from reached. Three comparisons no vs. immediate feedback, 

delayed vs. immediate feedback and immediate feedback vs. 

delayed feedback) were significant at the .05 le^^el. One other 
no vs. immediate feedback) approached the .10 level of significance. 

The comparison of between the quiz conditions was definitely 

not significant (p=.28). 

The results of Class 715 did not replicate those of Class 708. 
The 2x3 factorial analysis of variance is summarized in Table 5. 

None of the differences was significant at the .05 level. Most of the 
between groups variance appears to 'be contributed by the low mean of 



O 
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TABLE 3 

Part Correlations Under Different Treatments in Class 708 



Treatment 


’^F(Ai.Ac) 


^F (Ac .Ai) 


Immediate Qui^i 


.096 


.116 


Delayed Quiz 


.'209 


.118 


No Feedback 


.252 


.117 


Immediate Feedback 


-.133 


.380 


Delayed Feedback 


.290 


.004 



'9 



t 



TABLE 4 



Comparison of Part Correlations under Different Treatments in Class 708 



Comparison 



F(Ai.Ac) 

Delayed Quiz -Immediate Quiz 
No Feedback - Immediate 
Feedback 

Delayed Feedback-Immediate 
Feedback 

Delayed Feedback-No Feedback 



^F(Ac.Ai) 

Delayed Quiz -Immediate Quiz 

Immediate Feedback -No Feedback 

► 

Immediate Feedback-Delayed 
' Feedback 

No Feedback -Delayed Feedback 



z diff. 
r 



.1158 


.596 


.28 


.2410 


1.624 


.05 


.4324 


1.755 


.04 


.0411 


.171 


CD 

OO 

• 


.0021 


.011 


.99 * 


.2826 


1.173 


.12 


.3961 


1.609 


.05 


.1135 


.471 


.64 * 



* When no direction was predicted a two tailed test was used. 

All other comparisons are one tailed tests since direction was 
predicted. 



TABLE 5 



Results of Analysis of Variance on Final in Class 715 



Source 


SS 


df 


MS 


F 


Quiz 


3.95 


1 


3.95 





Feedback 


80.28 


2 


40.11 


1.47 


p >.10 


Q X F 


164.55 


2 


82.28 


3.02 


p >.05 


Within 


3625.69 


133 


27.26 


1.82 


p >.10 
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the section receiving the delayed quiz with no feedback. Since this 

section had the lovrest BTB and pretest score means, and the immediate 

# 

quiz sections had a higher BTB mean than did the delayed quiz sections 

(115.92 vs. 113.93) an analysis of covariance with respect to BTB and 

% 

pretest was applied. Those results are shown in Table 6. As can be 
seen the between groups variance was reduced by such an analysis as was 
the within groups variance, but still no significant differences obtain- 
ed. However, whereas in the non-ad jus ted means the mean of the delayed 
quiz sections was smaller than that of immediate quiz sections, in the 
adjusted means this was not true (see Table 7). 

There were two basic differences in Classes 708 and 715 besides 
the non -replication of the results. The total mean of the final exami- 
nation score was higher for the latter (36.14 compared to 33.14). All 
section means in Class 715 approached the highest scores in Class 708. 
Also, Class 715 was generally more intelligent than Class 708 and the 
difference in their BTB score means approached significance (t=1.63, 
df 251; t=1.65 at p=.05). 

Part correlations and the results of the correlation comparisons 

are shown in Tables 8 and 9 respectively. Generally, the comparisons 

follow the same pattern as did the results of Class 708 with two notable 

exceptions. The part correlation of Ai.Ac with the final was higher 

under the immediate quiz than under the delayed quiz sections contrary 

to prediction. However, this difference was not significant. Also, 

the difference in r„,.. . . between the no and immediate feedback sections 

F(Ai.Ac) 

failed to approach significance. Thus, three of five predicted differ- 
ences were supported at the .05 level of significance or below. The 

F(Ac .Ai)’ 



correlation, r 



appears to have been more stable across samples. 



r 



TABLE 6 





Results 


of Analysis of 


Covariance in 


Class 715 


Source 


SS' 


df 


MS' 


F 


Quiz 


.21 


1 


.21 


— 


Feedback 


18.68 


2 


9.34 


— 


Q X F 


59.15 


2 


29.58 


1.94 


error 


1997,59 


131 


15.25 





p >,10 









TABLE 7 



Means and 


Adjusted 


Means of 


Sections 


in 


Class 


715 


Section 


‘ E 


H 


D 


C 




A 


B 


Condition 


IN 


II ‘ 


ID 


DN 


DI 


DD 


Mean 


36.77 


37.50 


35.00 


33.92 


36 


.81 


37.45 


Adjusted. Mean 


36.20 


36.93 


35.42 


35.09 


36 


.18 


37.43 



Immediate Quiz 

Mean 36.31 



Delayed Quiz 
35.97 



Adjusted Mean 



36.09 



36.15 



TABLT-: 8 



Part Correlations Unde 


r D if To rent Tre 


atments in Class 715 


Treatnent 


^F (Ai.Ac) 


4 , 

^F(Ac.Ai) 


IiTur.ediate Quiz 


.293 


-.074 


Delayed Quiz 


.253 


.139 


No Feedback 


.226 


-.137 


Iinmcdiate Feedback 

I 

Delayed Feedback 


.143 


.395 


. 465 


-.186 ■ 



> 

C 
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Comparison of Part Corrclal io<u-. 
Comparison. 

F (Ai . Ac) 

Immediate Qui:<: -Delayed 


Under Di 

z diff 
r 


fferent Treatments 
S 

z -z 
r r 


in Class 7 
_P_ 


Quiz 

No F e e d b a c Ic - 1 mme d i a t e 


-.0432 


.247 


.81* 


Feedback 

De laycdi Feedback -Inraod fa te 


.0860 


.396. 


.35 


Feedback 

Delayed Feedbc;cl:-No 


.3597 


1.648 


.05 


Feedback 

^F'(Ac.Ai) 

D e 1 a }• e d Q u i z - 1 rm ■ le d i a t e 


.2737 


1.262 


.21* 


Quiz 

Inimcd ia te Feedback -No 


.2140 


1.224 


.22* 


Feedback 

Imn\odiate Feedback -Dt ’■ ayed 


.5556 


2.562 


.005 


Feedback 

No Feedback -Dt layed Feed- 


.6059 . 


2.777 


.003 


bacl: 


.0503 


.232 ■ 


. 82* 



i- 



UHien no diraction \-?as 
that pracUcLed a tv'O 
are one tailed tests 



predicted or if direction V7cnt 



contrary to 



tailed test V7as used. All other comparisons 
since direction \:as predicted. 
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The nii'.l i s of v.ir'. uh'o covjr i ni'.c-.' v/lLli ror.p»*cl: to ETB 



for tho CO 



‘ I - 



10 and 13 ror-pocti.Vu'iy 



Thi v.c; lu- 



. 0 ‘ 



bio 12. 'b'O rooult^ 



t 1 'io.c.o obt.sl'' ’ '.I s 70 vf. Only tlu: di'lv'y'.'il q'..n‘ r./ir;:.,- .d late frcJl’-.v’; 
and the cb.-l quLo/dolay-.d foMb.-o'; iicctlo’U-: contriiiai e slgni I'ic.-intly 

to the beL\;'-> o groop.c variance and arc significantly different fro.a the 
control group (DI , t-- 1 . 92 , df 78 , p <. 023 ). The adjusted means of the 
immedinte quiz sections are all lover than those of the delayed quin 
sections. Again Bartlett's test for homogoneity of variance s: ov/ed no 
significant difference, amon^ sec t ions . Although not significant, imme- 
diate feedback sections v;cre cvrisistently higher on final score than 
were the no and delayed fcedbacl: sections in all three samples. Also, 
the means of the no fecdbacl: conditions were consistently lower. 

Part correlaLlons for the co.;binod sample arc given in Table 13 . 

The coi.yvirlscn.s of thuse are sui::..:arir,C'.l in Table 14 . Because of the' 

larger nu-fjer of subjects and the fact tliat tlie correlations were Ion-, 
the dif ferc:,icu., liere probably are more infonaatlve as to the hypotliese."? 
proposed. Here again, one predict; or, is not suppert..-d (^p(Ai.Ac)’ 
layed quiz vs. immediate qul;-, p-:. 3 o), three are significant at approxL- 

niatclv the .01 level (i* ... . delayed feedback vs. Imdaedtate fecd- 

■' F(ivi .Ac) 

back , p- . 02 , and r,- / . ^ . ■ n immed int«,i £cccb,ick vs . no , p=- . 005 , and de - 

^ ^ ■’ t {exC , nl) 

# 

layed feed.back, p-.COl) and one is nut clear cut (^-'y (gp . ,\c) ’ fccd- 
bacl: VvS . im-.c-d isite fccd!>ack, p-.lO). All five In wh l.eh direction was 
predict-.' I n’er-, in th. it directi Those in which no d.’fferences were 



by 



’;v>t:n. 



i. ! fa'I.-d to cl'c:!- gn i fi cant differuncus 
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TABLE 10 





Results of Analysis 


of Variance 


for Combined 


Sample 


Source 


SS 


df 


MS 


F 


Quiz 


175.94 


1 


175.94 


\ 


Feedback 


86.79 


2 


43.40 


1.47 


Q X F 


89.13 


2 


44.57 


1.51 


Within 


7313.04 


247 


29.61 





* Significant at .05 level 
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TABLE 11 





Results of Analysis 


of Covariance 


for Combined 


Sampl 


Source 


SS 


df, 


MS 


F 


Qliiz ■ 


‘ 187.41 


1 


187.41 


7 .36* 


Feedback 


93.29 


2 


46.65 


1.83 


Q X F 


10.78 


2 


5.39 


.21 


Within 


6260.51 


246 


25.45 





* Significant at .01 level 
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TABLE 12 



Means and Adjusted Means of Combined Sample 



Condition 


IN 


II 


ID 


DN 


DI 


DD 


V C 


Mean 


34.19 


34.82 ■ 


33.61 


34.39 


36.50 


36.77 


34.22 


Adjusted Mean 


33.82 


34.82 


33.89 


34.83 


36.67 


36.16 


^ 9m mm 
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TABLE 13 

Par.t Correlations Under Different Treatments for Combined Sample 



Treatment 



•F(Ai.Ac^. 



■F(Ac .Ai) 



Immediate Quiz 
Delayed Quiz 



No Feedback 



Immediate Feedback 



Delayed Feedback 



.194 

.230 

.232 

-.035 

.360 



.072 

.123 

.010 

.390 

-.100 



TABLE 14 



Comparisons of Part Correlations Under Different Treatments 

for Combined Sample 



Comparison 

^F(Ai.Ac) 

Delayed Quiz-Immediate Quiz 

No Feedback-Immediate Feedback 

Delayed Feedback -Immediate 
Feedback 

Delayed Feedback -No Feedback 
^F(Ac.Ai) 

Delayed Quiz-Immediate Quiz 

Immediate Feedback-No Feedback 

Immediate Feedback -Delayed Feed- 
back 

No Feedback -Delayed Feedback 
* When no direction was predicted 



z diff 
r 


Zj-diff 


P 


.0377 


Szr-Zr 

.295 


00 


.2013 


1.281 


.10 


.3419 


1 

2.149 
' ( 


CVJ 

o 

• 


.1407 


\ 

.895 


.37* 



\ 



.0515 


.403 


.69* 


.4018 


2.556 

A 


.005 


.5121 


3.219 


' .001 


.1103 i 


.702 


• 48* 



a two tailed test was used. 



All other comparisons are one tailed tests since direction was 
predicted. 
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a type I error was made in Class 708 wherein the null hypothesis was 
rejected when it was in fact true. Coupled with this is the possibility 
that tlic three sections receiving the immediate quiz just happened by 
chance to be the poorest physics students of that particular class. The 
probability of making a type I error was, of course, .01. Out of a set 
of six groups, three can be combined, disregarding order, twenty ways. 
Thus, the probability of all immediate quiz groups combining with lower 
means with just chance operating would be 1/20 or .05. Including the 
control this probability would be 1/42 or .024. 

A second possibility stems from lack of control over the instruc- 
tor variable. It will be recalled that instructors were switched in the 
replication. If the instruction was contributing totally to the between 
quiz variance, then the delayed quiz conditions should have scored lower 
in the replication. This was not the case. However, if both the in- 
struction and the delayed quiz treatment was operating, then in the repli- 
" cation, these effects would have cancelled each other out. If this were 

so, one would expect that the means of Class 715 would be somewhere in 
between ‘ the immediate and delayed means of Class 708. This was not the 
case since the means of Class 715 were as high as the highest means of 
Class 708. 

A third explanation would be in terms of the higher intelligence 
level of Class 715. Noll (1939) pointed out that tests had a tendency 
to help students of lower intelligence levels, but not affect those of 
' higher intelligence. Class 715 could very well have been above the in- 
telligence level conducive to experimental effects. The ref lection "T)f 
higher final scores could be interpreted in this light. 

The fourth possibility influencing the replication is related 

i' « 

o 
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to the third and also would reflect overall higher scores. This has 
to do with the change in conditions of AMFU-A school. Class 708 was 
essentially a naive class in that it was the first class used for any 
experimentation for quite some time. However, in between 708 and 715 
every class was involved in some kind of research. Essentially, this 
made the instructors "research tired" and probably less conscientious 
about controls than they had been previously. In addition, the attri- 
tion via failure rate was rising and there was extra effort on the part 
of the school and instructors to teach the material. Such "added" 
treatments could \Vell have cancelled the effects of the experimental 
conditions found in Class 708. 

If the assumption is made that the diffevences found in Class 708 
are real differences, and are due to the influence of the treatments on 
mathemagenic behaviors then one must deal with the lack of support for 
the hypothesis that delay of informational feedback provides the same 
opportunity as the delayed quiz. One can interpret this that either the 
behaviors elicited are different or the material processed is different. 
\^at may be going on in the delay of feedback period is only a practice, 
rehearsal, or memorizing the questions on the quiz. Such would serve 
only to increase the effective number of trials on the quiz and may 
enhance a post-test score on the same questions. Another alternative 
is that the behaviors are the same but the material processed is diffe- 
rent. This can be seen diagramatically below: 

Condition ID: lecture/quiz/process quiz during delay/ 

infomational feedback 

\ 

Condition IQ: lecture/process lecture during delay/quiz/ 

infomation feedback 

What is being assumed here is that the student would behave mathemageni- 



cally with respect to the presentation immediately before the delay. 

In the immediate quiz/delayed feedback paradigm this would be the 
material on the quiz. In the delayed quiz paradigm the material of the 
lecture would be processed during the delay. Since the questions on 
the final were different than those on the daily quizzes, the mathema- 
genic behaviors of the delayed feedback pa.radigm would not be as perti- 
nent except in so far as the questions are directly transferable. 

The failure of r„... . . to be differentially reflected under 

F(Ai.Ac) 

immediate and delayed quiz conditions may be due to a design error. Each 
of these quiz samples included no, immediate, and delayed feedback condi- 
tions and the correlation could have reflected the average of these 
rather than any influence of the quiz condition. However, the quiz was 
varied independently in that the feedback conditions were equal in each 
quiz condition. 

On the other hand, Ai may not correlate differentially under the 

immediate and delayed quiz conditions just as the data implies. Thus, 

differential r , or r may reflect only the presence of struc- 

F(Ai.Ac) F(Ac.Ai) 

ture vs. non-structure and not be related to mathemagenic behavior. But 
an erroneous assumption is made here, namely, mathemagenic behavior does 
not go on under immediate quiz conditions. Such an assumption is con- 

’ i 

tradictory to the prediction of enhanced learning under delayed feedback . 

f 

conditions and the discussion above concerning it. Thus, what may be 
important is not the presence or absence of mathemagenic behaviors, but the 
facilitative or detrimental effects of such. In so far as mathemagenic 
behaviors are engaged in on the quiz material, if the criterion test is 
of the same type (i.e. a buoyancy problem with just the numbers changed) 
then Ai.Ac, assumed to include the engagement of mathemagenic behaviors, 



would correlate with criterion. However, the amount of positive trans- 



ference would be less or in other words, the mean of the immediate 
quiz groups final scores would be lower which was what was found. 

The conceptualization of the function of quiz and feedback is 
given below: 

I N: lecture/quiz/processing of quiz 

I D; lecture/quiz/processing of quiz/information as to 
behavior on quiz 

D N: lecture/process lecture/quiz, or test of mathemagenic 

behaviors utilized in taking quiz 

•D I: lecture/process lecture/quiz; same as D N/information 

as to correctness of mathemagenic behavior results 

D D: same as D I except the student has to wait for informational 

feedback 

In the immediate quiz conditions, the quiz thus functions in lieu of 
the person's own processing of the lecture material. Informational 
feedback would thus reinforce behavior on. the quiz or shortly thereafter 
if it is delayed. In so far as these are conducive to learning material 
inherent in the criterion final, then they will influence positively 
the final score. In the event that no feedback is given, then behaviors 

h 

detrimental to criterion, engaged in during the quiz, do not drop out 
unless the subject searches for or receives this information on his o^-m. 
Hence, the consistent, although statistically insignificant, lower 
means in the no feedback sections. " 

In the delayed quiz paradigm, the quiz serves as a means of uti- 
lizing the mathemagenic behaviors engaged in during the delay. So, 
.actually different kinds of behavior are being tested. Here again in- 
formational feedback serves to reinforce the behavior on the quiz, blit 

» 

in the delayed quiz this behavior is assumed to be more facilitative and 
generalizable . The non-fac.il itative behaviors utilized during the quiz 
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are dropped out with informationaj. feedback,- 

This study was only an exploratory one designed to assess 

V7hether something like mathemagenic beliaviors are inherent in the 
f 

learning of lecture materials as it is proposed to be in reading mate- 
rials. If one can accept that the results suggest that such is possible, 
two steps in further study are indicated. The first, of course, is a 
delineation of what constitutes these behaviors. The suggestions from 
reading material studies could be helpful here plus students own reports 
as to how they learn material. The second step is to study ways that 
these behaviors can be maximized in the classroom situation. T\ 7 o direc- 
tions are possible. (1) The lecture can be so organized to build into it 
the facilitative behaviors for the student. This is essentially what 
Rothkopf has done with reading materials (Rothkopf, 1965; Rothkopf & Coke, 
1963, 1966) and what Cronbach (1967) has proposed in dealing with indivi- 
dual differences. In other words, the lecture would be organized so 
that the relations intended would be drawn by the student and- the import- 
ant material picked out. Such an approach would be difficult, especially 
if individual differences prevail in students as well as in instructors. 
Another approach might be to teach mathemagenic behaviors in courses of 
study skills, listening, taking notes, reading, etc. with the assump- 
tion or hope that these would generalize to the behavior in the class- 

9 

room. In any case, the attempt would be made to understand how the 
organism is learning instead of the material he is learning. 

In line with the purpose and the results of this study the follow- 
ing conclusions are offered: 

1,. A short daily quiz administered one day after a lecture serves 
to evaluate for the student mathemagenic behaviors in which he engages 
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during the delay. Those behaviors which are facilitative to learning 
are reinforced whereas those which are detrimental to learning are 
eliminated. ^ 

2. When knowledge of results are delayed, mathemagenic behaviors 
are also engaged in, but it is the quiz material that is being processed 
rather than the lecture material. Hence, the transfer of learning to 

different questions on the final- examination is not as great as in the 
case of the delayed quiz. 

3. The utilization of mathemagenic behaviors by the student 
varies according to his achievement orientation. Persons with an 
independent orientation are more likely to engage in mathemagenic 
behaviors and rely on the^. own intrinsic evaluations. Persons with 

a dependent orientation are more likely to seek extrinsic guidance and 
evaluation in the form of immediate feedback and rely less on themselves. 
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